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1.  Introduction 

This  paper  deals  with  estimating  regression  coefficients  in  the  usual  linear 
model.  Let  y  be  a  T-component  random  vector  with  expected  value 

(1.1)  fiy  =  Z/L 

where  Z  is  a  T  x  p  matrix  of  numbers  and  ft  is  a ^-component  vector  of  para¬ 
meters.  (All  vectors  are  column  vectors.)  For  convenience  we  assume  that  the 
rank  of  Z  is  the  number  of  columns,  p.  The  covariance  matrix  of  y  is 

(1.2)  <?(y)  =  -  Zp)  (y  -  Zp)'  =  E. 

(Transposition  of  a  vector  or  matrix  is  denoted  by  a  prime.)  Again  for  con¬ 
venience  we  shall  assume  that  E  is  positive  definite.  The  problem  is  to  estimate 
P  on  the  basis  of  one  observation  on  y  when  Z  is  known. 

When  E  is  known  or  is  known  to  within  a  constant  multiple,  the  Markov  or 
Best  Linear  Unbiased  Estimate  (BLUE)  is  given  by 

(1.3)  b  =  (Z'E- 1 Z)- 1  Z'E~  *y. 

The  least  squares  estimate  is  given  by 

(1.4)  b*  =  (Z'Zr'Z'y. 

The  covariance  matrix  of  the  Markov  estimate  is 

(1.5)  #(b)  =  (Z'E- 1  Z)-  1 . 

The  covariance  matrix  of  the  least  squares  estimate  is 

(1.6)  #(b*)  =  (Z'Z)-1Z'EZ(Z'Z)-1. 

Both  of  the  estimates  are  linear  and  unbiased. 
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The  optimality  property  of  the  Markov  estimate  implies  that  #(b*)  —  #(b)  is 
positive  semidefinite ;  that  is.  any  linear  function  of  the  Markov  estimate  has  a 
variance  no  larger  than  the  variance  of  that  linear  function  of  the  least  squares 
estimate.  Since  the  least  squares  estimate  can  always  be  calculated,  but  the 
Markov  estimate  is  unavailable  if  the  covariance  matrix  £  is  not  known  to  within 
a  constant  of  proportionality,  an  interesting  problem  is  to  find  the  conditions 
under  which  the  least  squares  estimate  is  identical  to  the  Markov  estimate.  It 
will  be  noted  that  they  are  identical  when  £  is  a  multiple  of  the  identity.  I.  The 
general  answer  is  given  by  the  following  theorem. 

Theorem  1.  The  least  squares  estimate  (1.4)  is  identical  to  the  best  linear 
unbiased  estimate  (1.3)  if  and  only  if  Z  =  V*C.  where  the  p  columns  of  V*  are  p 
linearly  independent  characteristic  vectors  of  £  and  C  is  a  nonsingular  matrix. 

The  sufficiency  of  the  condition  was  essentially  given  by  myself  in  1948  in 
Skandinavisk  Aktuarietidskrift  [1],  In  that  paper  I  showed  that  if  y  is  normally 
distributed,  then  the  least  squares  estimate  is  identical  to  the  maximum  likelihood 
estimate  ;  under  normality,  of  course,  the  maximum  likelihood  estimate  is  best 
linear  unbiased.  Watson  [9]  studied  a  measure  of  efficiency  of  estimates  from 
which  the  necessity  of  the  condition  can  be  deduced ;  Magness  and  McGuire 
[7]  explicitly  deduced  the  necessity  (while  Zyskind  [11]  announced  it). 

A  problem  that  is  more  explicitly  and  specially  a  time  series  problem  occurs 
in  the  case  where  the  residuals  constitute  a  stationary  stochastic  process.  The 
property  ast  —  o{s  —  t),  where  £  =  (crst),  denotes  stationarity  in  the  wide  sense. 
In  general,  the  least  squares  estimate  and  the  best  linear  unbiased  estimate  will 
be  different.  The  characteristic  vectors  of  £  depend  on  the  values  of  the  serial  or 
lag  covariances  and  hence  the  best  linear  unbiased  estimate  depends  on  these 
parameters,  which  are  generally  unknown. 

In  this  case  we  consider  the  covariance  matrices  of  the  estimates,  normalize 
them  suitably  and  identically,  and  consider  the  limits  of  them  as  T  — ►  x. 
Grenander  in  [4],  Rosenblatt  in  the  Third  Berkeley  Symposium  [6].  and  these 
two  authors  in  [5]  found  conditions  for  which  the  two  limiting  covariance 
matrices  are  the  same.  They  did  not  indicate  that  their  results  were  asymptotic 
analogues  of  the  result  for  a  finite  sample,  and  the  statement  of  their  results 
and  their  methods  of  proof  do  not  make  it  easy  to  see  the  relationship. 

In  this  paper  I  shall  prove  the  results  for  the  finite  dimensional  case  and  the 
limiting  case  in  a  similar  fashion  in  order  that  the  relationship  between  the 
results  be  clearer  and  that  the  asymptotic  results  be  more  easily  understood. 
The  emphasis  here  is  on  the  linear  algebra :  the  rigorous  derivation  of  the  limits, 
which  is  rather  involved,  is  omitted  (but  is  given  in  Section  10.2  of  [3]). 

The  method  of  proof  is  not  the  most  direct  for  Theorem  1 .  because  the  proof 
uses  covariance  matrices  instead  of  the  structure  of  the  estimates  themselves. 
On  the  other  hand,  the  asymptotic  results  must  be  derived  in  terms  of  the 
covariance  matrices  because  the  order  of  the  observation  vector  increases,  and 
thus  the  structure  of  the  estimate  changes.  To  obtain  comparable  proofs, 
covariance  matrices  must  be  used  throughout.  A  byproduct  of  my  proof  of  the 
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theorems  is  a  different  statement  of  the  conditions  of  Grenander  and  Rosenblatt, 
which,  I  hope,  is  more  enlightening  than  the  original.  Watson  [9]  related  the 
two  sets  of  results  by  considering  the  finite  sample  case  in  the  framework  of  the 
approach  of  Grenander  and  Rosenblatt. 

2.  The  finite  sample  case 

We  shall  now  proceed  to  prove  Theorem  1  by  considering  the  conditions  for 
which  the  two  covariance  matrices,  (1.5)  and  (1.6).  are  identical.  To  study  this 
problem  it  will  be  convenient  to  transform  the  coordinate  system  in  the  T- 
dimensional  space  to  the  coordinate  system  defined  by  the  characteristic  vectors 
of  the  covariance  matrix  L.  Let 


where  ^  X2  =  '  '  '  =  ^r(>0)  are  the  characteristic  roots  of  L.  Let  V  be  a 
T  x  T  matrix  with  columns  as  corresponding  normalized  characteristic  vectors. 
These  properties  can  be  summarized  in  the  two  matrix  equations 

(2.2)  £V  =  VA, 

(2.3)  V  V  =  I. 

which  imply  L  =  VAV'  and  I  =  VV'.  We  can  refer  the  matrix  of  independent 
variables  to  this  coordinate  system.  Then 

(2.4)  Z  =  VG. 

where 

(2.5)  G'  =  gr>- 

and  gt  is  a  p-component  vector,  t  —  1,  •  •  •  .  T.  The  two  covariance  matrices 
depend  on  three  matrices  involving  Z  and  L.  These  can  be  written  in  terms  of 
A  and  G  as 

T 

(2.6)  Z'Z  =  G'V'VG  =  G  G  =  X  g(g;, 

t  =  i 

(2.7)  Z'EZ  =  G'V'LVG  =  GAG  =  £  A(g,g;. 

t=  i 

(2.8)  Z'L- 1  Z  =  G' V'L- 1  VG  =  GA  lG  =  £  yg,g). 

(=  i  K 

The  columns  of  V  are  characteristic  vectors  of  L  _  1  corresponding  to  roots  which 
are  the  reciprocals  of  the  characteristic  roots  of  2.  We  shall  follow  these  matrices 
along. 
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The  characteristic  roots  may  not  all  be  different.  Let  us  indicate  the  multi¬ 
plicity  of  the  roots  by  writing  the  diagonal  matrix  A  in  the  partitioned  form 


(2.9) 


(VlI  0  •  •  ■  0  \ 

0  v2I  0 

0  0  •••  v„I J 


where  Vj  >  v2  >  •  •  •  >  vH  (>0)  are  the  different  characteristic  roots.  The  orders 
of  the  diagonal  blocks  are  the  multiplicities  of  the  corresponding  roots,  say 
mx,  m2,  •  •  •  ,  mH  (£■%=  x  mh  =  T ).  We  partition  V  and  G  similarly. 


(2.10) 


(2.11) 


y  _  ^y(i)  y(2)  .  .  .  y(W)j 


G  = 


IA 


\G {H)/ 


Now  let  us  go  back  to  the  matrices  we  considered  previously,  and  express  them 
in  these  new  terms.  Z  is  written  as 


(2.12)  Z  =  f  \{h)G(h). 

h=  1 

The  three  matrices  appearing  in  the  covariance  matrices  are 

(2.13)  Z'Z  =  f  Gw'G(h), 

h=l 

H 


(2.14) 

(2.15) 


Z'LZ  =  X  vhGih)'G{h). 

h=l 

Z'2T 1  Z  =  f  -  G('I)'G,,I). 


The  definition  of  a  submatrix  of  V  may  have  some  indeterminacy  in  it.  We  can 
replace  \{h)  by  V(,I)Q(/,)  and  replace  G(/l)  by  Q,{h)'G{h\  where  Q(h)  is  an  orthogonal 
matrix  of  order  mh.  Such  a  transformation  leaves  each  of  the  last  four  equations 
invariant. 

Theorem  1  shall  be  shown  to  be  equivalent  to  the  following  theorem. 
Theorem  2.  #(b)  =  #(b*)  if  and  only  if 

(2.16)  I  P(G“>)  =  p, 

h=  1 

where  p( G(h))  denotes  the  rank  of  G(/I). 

In  order  to  simplify  the  study  of  the  conditions  for  the  equality  of  the 
covariance  matrices,  it  is  convenient  to  transform  the  matrices  again.  Let  P 
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be  a  nonsingular  matrix  such  that 

(2.17)  P'(Z'Z)P  =  I, 

(2.18)  P'fZ'EZJP  =  D, 

where  D  is  a  diagonal  matrix  with  cLXi  ^  d2 2  ^  =  dpp  >  0.  (These  are 

the  characteristic  roots  ofZ'EZ(Z'Z)-1.)  Let  us  also  make  the  transformation 
of  the  other  matrix,  P'(Z'L-1Z)P.  The  covariance  matrix  of  P-1b  is  the 
inverse  of  this  last  matrix.  The  covariance  matrix  of  P-1b*  is  D.  (This  can  be 
seen  from  the  original  expression  for  the  covariance  matrix  of  b*.  (1.6),  by 
multiplication  on  the  left  by  P_  1  and  on  the  right  by  P'~  1  and  with  use  of  the 
properties  of  the  matrices  we  have  just  discussed.)  The  question  of  equality  of 
the  original  covariance  matrices  has  now  been  reduced  to  the  problem  of  when 
the  covariance  matrix  of  P~  *b  is  D. 

The  three  matrices  in  ^(b)  and  ^(b*)  can  be  written 

H 

I  =  FZ'ZP  =  X  cW- 

h=  1 
H 

D  =  P'Z'EZP  =  X  vhC(h). 

h=  1 

H  1 

P' Z'L~ 1  ZP  =  y  -  C{h). 

h=  1  Vh 

where  Cw  —  P'G'^'G^P.  Note  that  p(Cw)  =  p(Gw).  Let  us  consider  the 
diagonal  elements  of  each  of  the  last  three  equations.  They  are 


(2.22) 

H 

1  =  I 

h=l 

r(h) 

• 

(2.23) 

II 

v  r(h) 
vhcii 

(2.24) 

H  1 

Z  1 

h=  1 

r(h) 

Wi  • 

Since  the  matrix  C(h)  is  positive  semidefinite,  each  diagonal  element  is  non¬ 
negative.  For  each  i  the  sum  of  these  nonnegative  components  is  1 ;  hence,  the 
elements  in  the  ith  diagonal  position  can  be  considered  as  probabilities.  Let 
Xt  be  a  random  variable  that  takes  on  the  value  vh  with  probability  c$\  h  = 
1,  •  •  •  ,  H.  Then  du  is  the  expected  value  of  this  random  variable.  The  last 
expression  is  the  expected  value  of  the  reciprocal  of  this  positive  random 
variable.  If  the  two  covariance  matrices  are  to  be  the  same,  the  ith  diagonal 
element  of  the  last  matrix  must  be  the  reciprocal  of  that  diagonal  element  of  the 
second  matrix.  Thus,  the  random  variable  just  defined  can  take  on  only  one 
value  with  probability  1.  (This  is  basically  the  condition  for  equality  in  the 
Cauchy-Schwarz  inequality.)  This  implies  that  for  each  i,  c-J1*  =  1  for  one  index 


(2.19) 

(2.20) 
(2.21) 
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h  and  is  0  for  other  values  of  h  because  the  vh  are  distinct.  These  facts  imply  that 
the  diagonal  elements  of  the  matrices  Cih)  are  1  ’s  and  0’s.  The  matrices  C(h)  have 
diagonal  elements  as  follows : 


If  a  matrix  C(h)  has  1  in  the  ith  diagonal  position,  the  other  matrices  have  0  in 
that  position.  (Then  dH  =  vh.  Since  the  vh  and  du  are  numbered  in  descending- 
order,  the  I  s  in  C(1)  are  in  the  upper  left  corner,  and  so  on.)  Some  matrices 
may  only  have  0’s  on  the  main  diagonal.  Since  C{h)  is  positive  semidefinite.  a 
diagonal  element  of  0  implies  that  the  entire  corresponding  row  and  column 
are  0.  Thus 


Since  the  Cw  sum  to  I,  and  the  nonzero  blocks  are  not  overlapping, 


(2.27) 


We  have  then  C(1)  with  an  identity  in  the  upper  left  corner  and  so  on.  The  rank 
of  each  Cw  is  equal  to  the  number  of  diagonal  elements  that  are  1.  Thus,  the 
sum  of  the  ranks  is  equal  to  p.  Therefore,  the  equality  of  the  covariance  matrices 
implies  that  the  sum  of  the  ranks  is  p. 

The  converse  can  be  obtained  by  use  of  Cochran's  theorem.  (See  Lemma  7.4.1 
of  [2],  for  example.)  However,  we  shall  use  a  simplified  proof  of  a  generalization 
of  one  part  of- Cochran’s  theorem  due  to  Styan  [8].  We  assume  the  sum  of  the 
ranks  of  the  C(,l)’s  is  p.  Let  the  nonnull  C(,l)'s  be  Lx .  *  •  *  ,  LK.  K  ^  p.  and  let  the 
ranks  of  these  matrices  be  rl ,  •  •  •  ,  rK,  respectively.  Then  L j  can  be  written  A'jAj. 
where  A  •  is  rj  x  p,j  =  1,  •  •  ,  K.  Let  U  be  the  diagonal  matrix  with  jth  diagonal 
block  of  order  rj  consisting  of  Ujl,  respectively,  where  Uj  is  the  jth  value  of 
v’i ,  •  •  •  ,  vH  corresponding  to  a  nonnull  C(h\  j  =  1 .  •  •  •  .  K.  Let 
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/  A.\ 

(2.28)  A  =  •' 

\Ak) 


Then  (2.19)  and  (2.20)  are 

(2.29)  I  =  A' A, 

(2.30)  D  =  A'UA. 

Equation  (2.29)  shows  that  A  is  orthogonal  as  Y* ,  =  p.  and  so  it  follows 

from  (2.30)  that  ^  ' 

(2.31)  D“ 1  =  A'U-1  A. 
which  is  (2.21).  Since 

(2.32)  £  p(  C(fc))=  X  0  =  p. 

h  =  1  j  =  1 

Theorem  2  is  proved.  (That  equality  of  covariance  matrices  implies  the  rank 
condition  can  be  proved  by  the  method  used  in  the  converse,  but  it  does  not 
generalize  directly  to  the  case  of  stationary  residuals.) 

As  was  indicated  earlier.  Gw  in  Z  =  YJ!=  i  V(,I)G(,,)  can  be  replaced  by 
q(/>)'qUi)  where  Q<h)  is  orthogonal.  In  particular.  Q(h)  can  be  chosen  so  that  G(,I) 
has  as  many  nonzero  rows  as  its  rank.  (For  the  nonnull  C{h)  or  G(/I),  the  resulting 
matrices  are  At .  •  •  •  .  AK.)  This  proves  Theorem  1  for  the  finite-dimensional  case. 


3.  Large  sample  theory  for  stationary  residuals 

We  now  turn  to  the  problem  involving  stationary  time  series.  The  elements  of 
the  covariance  matrix  of  y  are 


(3.1)  <r„  =  o(s  -  1)  =  f’  e«*-'ufiX)dX, 

J  ~  71 

where  f(X)  is  the  spectral  density,  which  is  assumed  to  exist.  Also  we  assume 
that  the  spectral  density  satisfies  the  inequalities 


(3.2) 


0  <  —  <  /(/ 1)  < 
2  n 


M 

2n 


when  m  and  M  are  some  positive  constants.  In  developing  the  asymptotic  theory, 
I  shall  not  attempt  to  state  all  of  the  conditions.  (They  are  given  in  Section  10.2.3 
of  [3].)  We  write 

1  00 

/W  -  f-  I 

h=  -  oo 


(3.3) 
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Let  the  diagonal  matrix  Dr  be  defined  by 
(3.4)  diag  (Z'Z)  =  diag  (Dx), 


where  we  take  the  positive  square  roots.  Since  we  are  interested  in  T  — *  oo,  we 
shall  use  the  index  T  when  convenient  to  emphasize  that  we  have  a  sequence  of 
estimates.  The  suitable  normalization  of  the  estimates  is  multiplication  by  this 
matrix  Dr.  We  consider  the  limits  of  the  covariance  matrices  of  Drb  and  of 
Dxb*.  The  question  is  what  are  necessary  and  sufficient  conditions  on  the 
independent  variables  and  the  spectral  density  such  that 

(3.5)  lim  ^(DTb)  =  lim  ^(Drb*). 

T  -*  oo  T  -»  x 

Let 

(3.6)  Z'  =  (Zj.---.Zj). 

Consider  the  sum  on  t  of  zt  +  hz't  and  multiply  on  each  side  by  Df  1  to  obtain  the 
matrix  of  lagged  correlations  of  order  h.  Let  the  limit  of  this  matrix  as  T  — ►  oo  be 

(3.7)  R(A)  =  lim  Df  1  V  zf  +  hz,'Df  1 . 

T-a o  , 

We  assume  that  these  limits  exist  for  t  =  0,  ±1.  ±2.  •  •  •  .  Then  this  sequence 
of  matrices  has  the  spectral  representation 

(3.8)  R(A)  =  J*  eahdM{X), 

where  M(A)  has  complex-valued  elements,  is  Hermitian,  and  has  increments  that 
are  positive  semidefinite. 

We  shall  now  consider  the  limits  of  the  covariance  matrices  of  the  normalized 
estimates.  Those  covariance  matrices  involve  the  limits  of  the  matrices 
Df 1  Z'  ZDf  Df  1  Z'LZDf  *,  and  Df  1  Z£~ 1  ZDf  1 .  In  fact. 

(3.9)  lim  ^(Dxb)  =  lim  (Df  1  Z'£- 1  ZDf  1 )- 1 , 

T  -*  oo  T -*  oo 

(3.10)  lim  <^(DTb*)  =  R_1(0)  lim  Df  1  Z'EZDf 1 R" 1  (0). 

T -*  oo  r-oo 


The  second  matrix  is 


(3.11)  lim  Dj1  X  Zzt  +  fcz^(^)Dr1  =  X  R(^)ff(^) 


h  =  —  (T  —  1)  t 


=  T  X  <y(h)eUh  dM(X) 

J  ~n  u  -  -  ~r 


2nf(X)  rfM(x). 
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Of  course,  these  operations  need  to  be  justified  to  give  a  rigorous  proof,  but  that 
requires  considerable  detail.  The  full  proof  is  given  in  Section  10.2.3  of  my  book 
[3]  and  is  along  the  lines  indicated  by  Grenander  and  Rosenblatt  [5].  The  three 
matrices  we  are  interested  in  can  be  written 


(3.12) 

lim  Df  1  Z'  ZDf  1  = 

r-oo 

(3.13) 

lim  D^Z'EZDf1  = 

T  -*  oo 

(3.14) 

lim  Df 1  Z'£_  1  ZDf 1  = 

T-+  oo 

jK  dM(A), 
jK  2n/(A)  dM(A), 


-n  2rc/(A) 


dM(A). 


The  derivation  for  the  third  matrix  is  an  involved  demonstration  also  given  in 
[3],  These  three  expressions  are  the  analogues  of  (2.6),  (2.7),  and  (2.8)  in  the 
finite-dimensional  case.  Carrying  the  analogy  to  the  finite-dimensional  case 
further,  we  shall  write  these  integrals  in  another  manner  to  resemble  (2.13), 

(2.14) ,  and  (2.15).  Let 

(3.15)  S(u)  =  {A|2te/(A)  ^  u},  m  g  u  g  M, 


(3.16) 


T(«)  =  f  dM(A) 

Js(u) 


The  component  functions  of  T(m)  are  real.  Then  our  three  matrices  can  be 


written  as 

lim  Df1  Z  ZDf  1  -  | 

'M 

(3.17) 

dT(u), 

r-oo  J 

,  ,  ( 

m 

'M 

(3.18) 

lim  Df  1  Z'EZDf  1  = 

u  dT{u), 

r-oo  J 

m 

(3.19) 

lim  Df  1  ZL" 1  ZDf  1  = 

'M  , 

-  dT(u). 

r-o° 

Jm  U 

Similar  to  the  finite-dimensional  case  we  let  P  be  a  nonsingular  matrix  such 
that 

(*M 

(3.20) 

P'  dT(u)  P  = 

J  m 

=  I, 

(3.21) 

P'  u  dT(u)  P  = 

J  m 

=  D. 

where  D  is 

diagonal  and  d{1  ^  d12  ^  ^  dpp  >  0.  The  same  transforma- 

tion  is  applied  to  the  third  matrix,  w-1  dM(«),  which  is  the  inverse  of 
lim^^  ^(Drb).  The  other  limiting  covariance  matrix  is  lim^^  ^(DTb*)  =  D. 
If  we  let 


(3.22) 


L  (u)  =  P'T(m)P, 
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then  the  three  matrices  of  interest  are 


(3.23) 

I  =  lim  P  Df  1  Z  ZDf1  P  = 

T->  oo  J 

(3.24) 

D  =  lim  P'Df  1  Z'EZDf  1  P  = 

T  -*  00  * 

(3.25) 

lim  P  Df  1  Z  L1  ZDf  1  P  = 

T~*  <x>  K  n 

M  j 

-  d\j{u). 


A  diagonal  element  of  L(w).  say  ffu).  has  the  properties  of  a  cumulative 
distribution  function.  The  corresponding  diagonal  elements  of  (3.24)  and  (3.25) 
are  u  dlu(u )  and  u~1  dlu(u),  which  are  the  expected  values  of  the  random 
variable  with  this  distribution  and  its  reciprocal.  Thus,  if  the  two  limiting 
covariance  matrices  are  equal,  the  matrix  (3.25)  is  the  inverse  of  (3.24)  and 


(3.26) 


u  dlH(u) 


m 


1 

-  dlu(u) 
u 


this  implies  that  lu(u)  has  one  point  of  increase  and  the  increase  is  1  at  this 
point.  Let  the  points  of  increase  be  ^  u2  •  •  •  ^  uK  ^  0,  and  let  Ly  be  the 
increase  of  L (u)  at  Uj.  j  —  1,  •  •  •  ,  K.  Then  the  three  matrices  can  be  written 

K 

(3.27)  1=  lim  P'  Df  1  Z'  ZDf  1  P  =  £  L, 

T-*  00  j  =  1 

K 

(3.28)  D  =  lim  P' Df  1  Z'LZDf  1  P  =  £  WyLy, 

T->co  j=1 

K  1 

(3.29)  lim  P'Df  1  Z'L"  1  ZDf  1  P  =  X  ~Lj- 

T  -*  oo  j=  i  Uj 

We  are  now  back  to  the  same  forms  that  we  had  for  the  finite-dimensional  case, 
(2.19).  (2.20).  (2.21).  The  only  difference  is  that  in  the  earlier  case  we  had  not 
culled  out  the  vacuous  matrices  C(h).  From  this  point  on  the  reasoning  is  the 
same.  The  matrices  L1.L2.-’'.LK  have  the  form  of  (2.27);  that  is,  the 
diagonal  blocks  are  I  s  and  0  s  and  off  diagonal  blocks  are  0  s. 

The  converse  is  similar  to  the  finite-dimensional  case.  If  L (u)  has  K  points  of 
increase  and  the  sum  of  the  ranks  of  the  increases  is  p  (and  the  increases  are 
positive  semidefinite  with  sum  of  I  and  weighted  sum  of  D),  then  by  the 
previous  reasoning,  they  are  of  the  form  (2.27)  and  (3.29)  is  D~  1 .  We  put  these 
properties  in  terms  of  M(A)  and  summarize  them  in  a  theorem. 

Theorem  3.  The  limiting  covariances  of  Drb  and  Drb*  are  identical  if  and 
only  iff(l)  takes  on  no  more  than  p  values  on  the  set  of  k  for  which  M(A)  increases 
and  the  sum  of  the  ranks  of  j  dM(A)  over  the  sets  of  k  for  which  f  (A)  takes  on  these 
values  is  p. 
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The  set  of  A  for  which  M(A)  increases  is  called  the  spectrum  of  M(A).  The  sets  of 
A  for  which  /(A)  assumes  its  values  are  called  the  elements  of  the  spectrum.  The 
properties  of  Lt,  •  •  •  ,  LK  (idempotent  and  orthogonal)  determine  these  sets; 
Grenander  and  Rosenblatt  used  them,  though  indirectly. 

When  the  residuals  are  uncorrelated, /(A)  =  a{0)/{2n)  and  the  conditions  of 
Theorem  3  are  satisfied.  However,  we  may  be  interested  in  conditions  on  the 
independent  variables  also  which  insure  that  least  squares  be  asymptotically 
efficient  regardless  of f(k). 

Theorem  4.  The  limiting  covariances  of  Drb  and  Drb*  are  identical  for  all 
stationary  processes  with  spectral  densities  which  are  bounded  and  bounded  away 
from  0  if  and  only  if  M(A)  increases  at  not  more  than  p  values  of  k.  0  ^  k  ^  n.  and 
the  sum  of  the  ranks  of  the  increase  in  M(A)  is  p. 

If  the  number  of  points  at  which  M(A)  increases  is  at  mostp,  0  ^  k  5$  n,  then 
the  spectrum  of  M(A)  consists  of  these  p  points  and  their  corresponding  negative 
values.  The  spectral  density  (which  is  symmetric)  can  then  take  on  at  most  p 
values,  namely,  its  values  at  these  p  points,  0  5^  k  ^  n.  On  the  other  hand  if 
M(A)  increases  at  more  than  p  points,  0  ^  k  52  n  then  an  f(k)  can  be  constructed 
so  that  it  takes  on  no  more  than  p  values. 

An  example  of  independent  variables  {zjt}  such  that  M(A)  has  one  point  of 
increase  is  zjt  —  tj~l,j  =  1,  •  •  •  ,p,  t  —  1,  2,  •  •  ;  the  jump  is  at  0  and  the 
increase  in  M(A)  at  A  =  0  is  a  positive  definite  matrix 


(3.30) 


M0 


/(2j  -  l)1'2  (2 k  -  l)1/2\ 
V  j  +  k  -  1  / 


In  this  case  R {h)  =  M0,  h  =  0,  ±  1,  •  •  •  .  If 

H 

(3.31)  zt  =  a0  +  Yj  («/  cos  V  +  Pj  sin  Vjt). 

j=  i 

then  M(A)  has  an  increase  of  rank  1  at  A  =  0  and  an  increase  of  rank  2  at 
A  =  Vj  (with  0  <  Vj  <  n).  j  =  1,  •  In  these  examples  the  spectral  distri¬ 

bution  function  of  each  independent  variable  is  a  pure  jump  function,  which 
can  be  considered  as  the  opposite  of  a  density.  Trigonometric  functions  act  like 
characteristic  vectors  of  a  covariance  matrix  in  the  sense  that  they  are  involved 
in  spectral  representation.  Comparison  of  £  =  VAV'  and  (3.1)  suggests  that 
columns  of  V  correspond  to  functions  e1/s,  the  diagonal  components  of  A 
correspond  to  the  values  of  2nf(k),  and  summation  with  respect  to  the  index  of 
diagonal  components  of  A  corresponds  to  integration  with  respect  to  k/{2n). 
The  analogue  of  V'EV  =  A  is  (3.3),  which  involves  a  limiting  procedure. 

o  o  o  o  o 

The  author  is  indebted  to  George  Styan  for  helpful  discussions. 
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